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ABSTRACT 

Laboratory Earth, a series of three NASA-Sponsored, on-line graduate courses for K-8 teachers, was designed to meet a 
variety of learning styles and appeal to teachers' motivation to learn the content and improve their teaching. This is 
especially important to teachers as they seek to demonstrate "highly qualified" status to meet No Child Left Behind 
standards. These graduate-level courses consist of four modules of two to four lessons each. Pre- and post-course 
surveys indicated significant increases in teachers' (n=51) content knowledge, science teaching efficacy beliefs (STEBI- 
A), sense of community within the course (LEO) and science teaching enjoyment (STES). Qualitative data indicated 
teachers valued the cohort system, content aligned to teaching needs, and the instructor's response to requested 
feedback. Results indicated that online courses can provide valuable professional development opportunities for K-12 
science teachers to deepen their knowledge, sharpen their skills, and maintain their knowledge of science developments. 
Because teachers play an important role in the development of their student's attitudes towards science, it is extremely 
important that science and education communities collaborate to create courses that use contemporary pedagogy to 
address the content-knowledge needs of teachers required by National Science Standards criteria. 


INTRODUCTION 

Through the Nebraska Earth Systems Education 
Network (Gosselin and others, 1995, 1996, 1999, 2002), we 
have worked to design professional development 
opportunities for pre- and in-service K-12 educators 
related to Earth system science that are learner-, 
knowledge-, assessment-, and community-centered as 
advocated by Bransford et al. (2000). Over the past few 
years, we have transitioned much of our professional 
development activities from traditional, face-to-face 
environments to the on-line environment. A recent report 
on online learning (DOE, 2009) indicates that the on-line 
environment is an effective educational option across 
different content and learner types for both 

undergraduate and graduate students as well as 
professionals in a wide range of academic and 
professional disciplines. 

Not only are online courses effective, they are an 
attractive professional development option for educators, 
who are place-based or have travel and time limitations. 
These limitations are especially prevalent in much of the 
Great Plains region and the western United States where 
many rural teachers live as far as 200 miles from an 
institution of higher education. In an open-ended 
response survey item, 211 respondents (29%) 

spontaneously described that they desired online 
professional development. As this was unprompted 
specifically by the survey, the authors argued that the 
actual fraction of teachers who might take advantage of 
online professional development is likely much higher 
(Slater and others, 2009). To meet these needs, we initiated 
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the Laboratory Earth on-line, distance-delivered 
professional development series to improve K-12 
educator's knowledge, understanding of content 
connections, and ability to teach science in the context of 
the Earth as a system. 

One of the major goals of Laboratory Earth is to help 
improve teacher's content knowledge through the use of 
effective science teaching methods, which Cox and 
Carpenter (1991) aggressively argued is required to 
improve science teachers' skills. The purpose of this paper 
is two-fold. First, we present the course design and 
assessment approach used for the courses in the 
Laboratory Earth series as a potential model for other 
instructors. Second, we specifically focus on data collected 
to address the following questions from one of the 
Laboratory Earth courses. Earth's Natural Resource 
Systems (or Lab Earth 2) as measures of successful online 
K-12 teacher professional development: 

1. Does the Lab Earth approach achieve the cognitive 
goal of increasing teachers' knowledge and 
understanding of Earth science content? and 

2. Does the Lab Earth approach improve teacher's self 
efficacy and beliefs about their ability and 
confidence to teach science in the context of the 
Earth as a system? 

COURSE DESIGN 

The Laboratory Earth course series currently consists 
of three, graduate-level, distance-delivered, online courses 
(Table 1). All three courses are designed to use a learner-, 
knowledge-, assessment-, and community-centered 
approach that is consistent with research on learning 
(Bransford and others, 2000; Manduca, 2007). The content 
for Laboratory Earth is organized according to the 
National Science Standards (NRC, 1996) criteria for what 
"must" be taught and what a person should know by the 
completion of high school. All the courses share the 
common vision and what we refer to as the I 2 A 
philosophy, in which we use an Inquiry approach, 
emphasize the Integration of scientific disciplines, and the 
Application of content to the world outside the classroom. 
Table 6 gives examples of two of the content mastery 
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prompts that depict the I 2 A philosophy. In both of these 
prompts, students use what they learned through the 
inquiry-based module activities to integrate and apply the 
information to a new situation. Laboratory Earth course 
content is also presented in the context of the Earth as a 
system following the overarching theme that "Everything 
is Connected to Everything Else." Our course design 
provides a framework to help teachers understand the 
concept of systems and connect the concepts of energy 
and matter to the scientific phenomena they experience 
daily. 

Participants use the BlackBoard™ course management 


system via the Internet to access the course modules and 
to interact with other participant teachers and course 
instructors. To be sure that all participants have access to 
the software required, we established a download center 
where participants could access Adobe, Microsoft, and 
other products they might need. Participants' need to 
have Internet access and basic technology skills; such as 
familiarity with a Web browser, e-mail, word processing, 
and Internet searching. 

With regard to online learning, these courses were not 
asynchronous, independent study experiences. Rather, 
participants reflected on their learning through online 


TABLE 1. SUMMARY OF MODULES IN LABORATORY EARTH COURSESi 



Lab Earth 1: Earth and its 
Systems 

Lab Earth 2: Earth's Natural 
Resource Systems 

Lab Earth 3: Earth's Changing 
Systems 

Module 1 

Earth's Spheres: Linking 
Science, Systems, and 

Society 

Natural Resources and 
Civilization 

From the Universe to the 

Earth and Everything in 
Between 


In addition to the 
opportunity to interact with 
your new colleagues, we will 
introduce you to the general 
characteristics of the four 
major subsystems, or spheres, 
of the Earth system. 

Investigate what are natural 
resources and how they are 
part of a larger system. The 
systems approach recognizes 
that humans are dependent 
on, impact the distribution of, 
and influence natural 
resource systems. 

Investigate how the Earth has 
changed through time and 
space using a systems 
approach to understanding 
our Earth and that everything 
is connected to everything else 
(ECEE). 

Module 2 

Earth's Matter, and their 
Interaction 

Rock and Mineral Resource 
Systems 

Changes through Geologic 
Time 


Topics include mass, density, 
weight, energy and heat 
transfer, convection, 
radiation, and the interaction 
between energy and matter. 
Focus on providing examples 
of where these concepts and 
principles are important in 
the Earth system. 

Explore basic concepts and 
principles related to rocks 
and minerals as natural 
resources for many of the 
materials that we use daily. 
This module focuses on the 
natural resources that we 
derive from the geosphere. 

Focus on the application of 
basic concepts and principles 
to reading the rock record. 

Module 3 

Earth in Space 

Soil Resource Systems 

Cycling in the Earth System 


Explore how things move in 
the sky, the nature of our 
solar system's planets, some 
of the major misconceptions 
about astronomy, and collect 
and analyze astronomical 
data. 

Investigate the properties of 
soil and their relationship to 
the soil forming factors. 
Understand the importance 
of soil as a natural resource. 

Examine the important role 
that cycles play in the Earth 
Systems resulting from an 
exchange of mass and energy 
between Earth's four major 
spheres and related 
subsystems. 

Module 4 

Weather and Climate 

Water Resource Systems 

The Environment: Yours, 

Mine, and Ours 


Examine the characteristics of 
the atmosphere and processes 
that influence the movement 
and transfer of energy and 
matter, specifically water, in 
the Earth's atmosphere. 
Designed to improve 
understanding of basic 
concepts related to weather 
and climate. 

Explore the processes by 
which water moves through 
the water cycle. Use the 
hydrologic equation to 
understand the importance 
and limitation of water 
resources as well as strategies 
for sustainable use. 

Investigate a range of 
challenges that humans have 
created as they have interacted 
with the local and global 
environment. 


1 The full syllabi of all the Laboratory Earth courses can be viewed at nesen.unl.edu 
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discussion and networking with each other. This type of 
feedback and interaction promotes self-regulation as 
students are able to see their own progress toward their 
learning goals and where they still need to add 
understanding. The use of the discussion board allows 
students to evaluate their own understanding compared 
to their peers. The discussion board also allows students 
to pose questions in a safe environment. Reflection and 
self-regulation are important to metacognition and are 
recognized as major factors in the effective use of on-line 
learning environments (Schwerin and others, 2006, 
Waterhouse, 2005, Keller and Slater, 2003b). Laboratory 
Earth pedagogy encourages participants' intrinsic 
motivation to learn the course material rather than look to 
external motivators (such as grades and points) that can 
detract from authentic, personal learning. 

As we have developed and implemented the Lab 
Earth courses, we have become increasingly committed to 
taking advantage of the strengths of online delivery that 
contribute to content knowledge enhancement and 
promote communication and collaboration (Waterhouse, 
2005; Keller and Slater, 2003a; 2003b; Prather and Slater, 
2002). The design of the Lab Earth program focuses on a 
context-based approach in which context and application 
of science are used as the starting point for the 
development of scientific ideas (Bennett, Lubben and 
Hogarth, 2006). For example, in the water resource 
module, students begin the module through the 

TABLE 2. EXAMPLE FROM LAB EARTH 2 OF 
MODULE STRUCTURE INCLUDING LESSONS 
AND ACTIVITIES! 

Module 1. Introduction - Natural Resources: Linking Science, Society 
and Systems 

Lesson 1: Welcome to Earth's Natural Resource Systems 

1. Activity 1-1 Introduce Yourself 

2. Activity 1-2 Expectations: Yours, Mine and Ours 

3. Activity 1-3 What is Your Learning Style Preference? 

4. Activity 1-4 Discuss the Implications of Learning Styles 

Lesson 2: Natural Resources and Civilization 

1. Narrated PowerPoint: Natural Resources and Civiliza¬ 
tion 

2. Activity 1-5 What Materials Do I Use Each Day? 

3. Websites for Activity 1.6 

4. Activity 1-6 Natural Resources: Developing a com¬ 
mon language 

Lesson 3: Earth's Spherical Systems 

1. Narrated PowerPoint - Earth's Spherical Systems I 
Activity 1-7 Website Research 
Narrated PowerPoint — Earth's Spherical Systems II 
Reading 1 - "Why an Earth Systems Approach?" 

Reading 2 - "Spaceship Earth" 

Reading 3 - "Earth's Spheres" 

Activity 1-8 ECEE Discussion 

Content Mastery Activity 


1 The full syllabus of Laboratory Earth: Earth's Natural 
Resource Systems can be viewed at nesen.unl.edu 


assessment of their own local water use using a water 
diary. These data are then compared to other student 
usage patterns. This is followed by an investigation of 
where their water resources originate. In the rock and 
mineral resources module, each person records all the 
materials they use throughout the day from toothpaste to 
computers. This is followed by an investigation of the 
ingredients of individual materials in terms of the source 
of the ingredients and from where they are mined. 

For each course, content materials are organized into 
four modules that include two to four lessons and 
multiple activities (Table 2). The Lab Earth courses use a 
continuum of strategies to meet multiple learning styles 
(Gardner, 1992; Wang, Wang, Wang, and Huang, 2006) 
that help participants successfully meet the learning 
outcomes. This approach recognizes that participants have 
multiple learning styles and diverse ways to take-in and 
give-out information. These strategies include direct 
instruction in the form of voiced-over PowerPoint 
presentations, group discussions and projects, field-based 
activities, kitchen labs, simulated experiments, individual 
reading assignments, online writing responses to open- 
ended questions, and inquiry-based activities that result in 
students constructing new cognitive understanding. Visit 
the NESEN website (nesen.unl.edu) for access to examples 
of specific methodologies and activities. Assignment 
deadlines encourage participants to progress 
synchronously through the course, but also allow 
individuals to work asynchronously as their schedules 
allow. 

In the Lab Earth courses, we want participants to 
focus on learning, not grades. Unfortunately, grades have 
to be issued. Past experiences with teachers taught us that 
they are often quite anxious about the assignment of 
course grades. To reduce anxiety, our grading strategy has 
evolved to a system that use, what we call, content 
mastery assignments (CMAs) to document participant's 
mastery of course content. To help everyone acquire the 
required concept knowledge and understanding, we use 
an iterative grading system that monitors active 
participation and on-time, quality completion of 
assignments and allows participants to revisit and 
resubmit their assignments and CMAs until they are 
satisfied with their performance level (i.e., grade). 

Although we do not explicitly teach metacognitive 
skills (planning, monitoring, and evaluating), we provide 
an opportunity for students to take control, become more 
confident, independent learners, and employ higher-order 
skills through the CMAs, Through CMAs, students 
indirectly use the basic elements of metacognition to 
achieve the cognitive goal of learning Earth science 
concepts. According to Livingston (1997), cognitive and 
metacognitive strategies are closely intertwined and 
dependent upon each other. CMAs promote the use of 
learner-based reflection that available research indicates 
are effective and improve the learning outcomes of 
students when they pursue learning as individuals in an 
online environment (Bixler, 2008; Chang, 2007; Chung, 
Chung, and Severance, 1999; Crippen and Earl, 2007; 
DOE, 2009, Nelson, 2007; Saito and Miwa, 2007; Shen, Lee, 
and Tsai, 2007). 
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TABLE 3. EXPECTED LEARNING OUTCOMES FOR LABORATORY EARTH 2, EARTH'S NATURAL RESOURCE 
SYSTEMS 

1. Describe and explain the basic interactions between the hydrosphere, geosphere, atmosphere, and biosphere. 

2. Acknowledge and work with individuals who have different perspectives about natural resources. 

3. Develop conceptual models for a variety of Earth's natural resource systems that qualitatively include mass and 
energy exchange. 

4. Demonstrate an understanding of the properties, occurrence and distribution of water and soils. 

5. Demonstrate an understanding of rocks and minerals as fundamental Earth resources. 

6. Explain the basic chemical and physical processes that control the distribution of Earth's natural resources. 

7. Explain the social and economic issues that control the availability of mineral and energy resources. 

8. Collect basic data required for the analysis of natural resource systems. 

9. Analyze and interpret graphs. 

10. Understand the dependence of all people on both renewable and non-renewable resources. 

11. Describe the impact of humans as stewards, managers and components of natural resources systems. 

12. Demonstrate an understanding about natural resources on other planetary bodies. 


RESEARCH DESIGN 

We used an action research approach (Sagor, 1993; 
Dick 2000) in which we collected data throughout the 
course that informed our instructional approach and 
provided the basis for making changes as the courses 
evolved. A mixed-methods research design (Creswell, 
2003) simultaneously collected quantitative and 
qualitative data. Qualitative data and participants' 
response to feedback prompts, helped illuminate the 
quantitative results. Regular meetings (throughout the 
course duration) among the course instructors (science 
professor, science education professor, and doctoral 
students in science education) allowed for reflective and 
formative deliberation on course content and methods. 

To address our first question, "Does the Lab Earth 
approach achieve the cognitive goal of increasing teachers' 
knowledge and understanding of Earth science content?", 
we used content questions and content mastery 
assignments to assess the extent to which participant 
teachers' understand and apply Earth science concepts as 
defined by the course's learning objectives. Table 3 


provides the learning objectives for Lab Earth 2. Question 
two, "Does the Lab Earth approach improve teacher's self 
efficacy and beliefs about their ability and confidence to 
teach science in the context of the Earth as a system?", was 
addressed using the Science Teaching Efficacy Beliefs 
Instrument (STEBI-A) (Riggs and Enochs, 1990). The 
Beliefs About Science (BAS) scale was used to assess the 
degree to which the participant teachers enjoyed teaching 
science. We also assessed the degree to which participant 
teachers felt a sense of belonging and community with the 
Laboratory Earth Orientation (LEO) scale. The BAS and 
the LEO scales were created by Lab Earth instructors. 

PARTICIPANT TEACHERS 

Fifty-one teachers were enrolled in the Lab Earth 2 
course as part of four cohorts. This group of teachers 
included 31 teachers in Cohorts 1 and 2 (8-week course 
format) and 19 teachers in Cohorts 3 and 4 (16-week 
course format). The course materials were designed for 
adult learners that included K-8 and some high school 
teachers. Pre-service teachers were also allowed to 


TABLE 4. LAB EARTH 2 PARTICIPANT TEACHER DEMOGRAPHICS 


Term 

N 

Gender 

School 

Teaching Level 

Weekly Hours of 
Science Teaching 

Total Years of 
Experience 

Median 

Avg. 

Median 

Avg. 

Spring 

2007 

19 

17 Female 

2 Male 

5 Elementary 

8 Middle 

6 High 

15 

18 

10 

12 

Summer 

2007 

12 

8 Female 

4 Male 

5 Middle 

5 High 

2 Preservice 

10 

15 

2 

7 

Fall 

2007 

7 

3 Female 

3 Male 

3 High 

4 Preservice 

0 

14 

2.5 

3 

Spring 

2008 

13 

10 Female 
3Male 

4 Middle 

8 High 

1 Preservice 

25 

22 

6 

8 
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TABLE 5. EXAMPLES OF CONTENT QUESTIONS FROM LAB EARTH 2 _ 

What is soil? How do the soil forming factors influence how rocks become soil? How do the soil forming factors 
influence the landscape? 

How are groundwater and surface water related? How do human activities potentially influence this relationship and 
how do we know? 


participate in the course. Participant demographics are 
provided in Table 4. 

RESEARCH TOOLS AND DATA COLLECTION 

Pre- and post-course surveys measured teachers' 
content knowledge, science teaching efficacy beliefs, 
science teaching enjoyment, and course cohesion. 
Feedback prompts elicited participant teachers' evaluative 
remarks at the end of each module and at the end of the 
course. The following descriptions define the instruments 
used in the assessment of these courses. 

Content knowledge assessment 

Content Questions and Content Mastery Assignments 
determined the extent to which the teachers learned Lab 
Earth 2 content as defined by the learning objectives 
(Table 3). The course instructor developed content 
questions based on the Earth Science Content guidelines 
for grades K-12 (American Geological Institute, 1991). 
Twelve, single-to-multiple-part questions were intended 
to measure teachers' content knowledge. Table 5 provides 
examples to illustrate the depth and breadth of knowledge 
solicited by the content questions. 

Evaluation of the content questions provided pre- and 
post-course measures of participant teachers' content 
knowledge, integration and organization of content and 
use of resources. A key component of this content 
assessment was to ascertain the levels of content 
understanding as delineated in Bloom's Taxonomy 
(Bloom, 1994). An assessment rubric generated teachers' 
scores ranging from zero to two for each of the 12 
questions . A score of zero indicated an incorrect response 
or no response provided. A score of one indicated the 
response reflected factual knowledge (lower cognitive 
level response such as a simple definition of the concept); 
and a score of two indicated the response primarily 
reflected both knowledge and comprehension (higher 
level of cognition such as applying the knowledge to a 
new situation, circumstance, or context). 

In Cohorts 1 and 2, content questions were used 


exclusively to assess content knowledge. Participants were 
asked to revisit the questions after each module. Feedback 
from participants, an examination of the work quality for 
individual activities and modules, and the nature and 
extent of responses on the course discussion board 
indicated that participants and the instructional team 
needed an additional tool to assess the extent to which 
learning was taking place related to the specific learning 
objectives for individual modules. For cohorts 3 and 4, 
Content Mastery Assignments were developed to assess 
content learning within each module as defined by the 
learning objectives for the module. The CMAs are 
provided at the beginning of each module. In theory, this 
provides the students opportunity to use their 
metacognitive skills to plan their approaches to complete 
the task, monitor their learning in the context of their plan 
as the module progresses, and evaluate their level of 
understanding after the completion of the task based on 
instructor feedback. CMAs consist of a scenario that 
presents a problem or situation and a rubric that outlines 
the general expectations for their response (see Table 6). 
Students choose and plan their own presentation 
approach and format to synthesize their knowledge and 
understanding of a module's material and to address the 
scenario. Presentation formats included traditional essays, 
PowerPoint presentations, newsletters, newspaper 
articles, concept maps, poetry, photo essays, and movies. 
If the materials presented do not address the elements of 
the rubric, the student is asked to reevaluate, revisit and 
resubmit the activity. The student can resubmit as many 
times as they want. The goal is for the student to acquire 
the required concept knowledge and understanding. 

Science teaching efficacy beliefs and enjoyment of 
science teaching 

The Science Teaching Efficacy Beliefs Instrument 
(STEBI-A) (Riggs and Enochs, 1990) assessed teachers' 
beliefs toward science teaching and learning. The STEBI-A 
includes 25 items measured on a five point Likert scale 
ranging from strongly agree to strongly disagree. Scoring 


TABLE 6. AN EXAMPLE OF A CONTENT MASTERY ACTIVITY PROMPT FOR LAB EARTH 2 _ 

Module 1. The history of human exploration of the Earth has been driven by the quest for wealth and prosperity de¬ 
rived from the elements obtained from rocks and minerals. We use resources from rocks and minerals every day, but 
these resources are nonrenewable. In the context of your understanding of rocks and minerals as resources, address the 
following: 1. How has the quest for strategic minerals influenced human history? Provide, at least, one specific example 
(You cannot use salt!). Be sure to explain what a strategic mineral or material is. 2. What economic factors illustrated in 
Cookie Mining do you think will have the greatest influence on the future pursuit of rock and mineral resources? It is 
very important to provide justification for your answer. 

Module 2. Every rock tells a story. Rocks and minerals also record the history of the Earth. Our challenge is to learn 
how to read the clues that rocks provide. 1. Provide an explanation of how the characteristics of igneous, metamorphic, 
and sedimentary rocks can be used to tell the history of an area. Provide specific examples for each rock type. 2. What is 
the theory of plate tectonics? How does this theory help explain the relationships between the three major rock types 
(igneous, metamorphic, and sedimentary) and the rock cycle. 
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TABLE 7. THE BELIEFS ABOUT SCIENCE (BAS) 

SCALE (TABLE 7)i _ 

1.1 enjoy teaching science. 

2. The time that I spend teaching science is time well spent. 

3.1 always look forward to teaching my next science les¬ 
son. 

4.1 like teaching science as much as any other subject. 

'Participants responded using a five point Likert format (1 = low enjoy¬ 
ment and 5 = high enjoyment.) See text for discussion. 

was accomplished by assigning a score of 5 to the 
positively phrased items (strongly agree) and a 1 to the 
negatively worded items (strongly disagree). STEBI-A 
determines teacher beliefs according to two subscales: 
Science Teaching Outcome Expectancies (STOE) and 
Personal Science Teaching Efficacy beliefs (PSTE). The 
Beliefs About Science (BAS) scale (Table 7) was written by 
the course instructor to assess the degree to which the 
participant teachers enjoyed teaching science. The scale 
included 4 items that used a five point Likert format (1 = 
low enjoyment and 5 = high enjoyment.) 

Course cohesion 

The Laboratory Earth Orientation (LEO) scale (Table 
8) was developed by the course instructor to assess the 
degree to which participant teachers felt a sense of 
belonging and community within the Laboratory Earth 
Sciences course. The scale included 6 items that use a five 
point Likert format (1 = low cohesion and 5 = high 
cohesion). 

Formative feedback prompts 

Feedback prompts posted to BlackBoard (after the 
completion of each module) helped to generate insights 
into the ways in which teachers' measured the benefits of 
their Laboratory Earth course experiences. These prompts 
provided formative assessment throughout the course 
development over four semesters: What is going well? 
What could have gone better? What suggestions do you 
have to improve the module? Thus, teachers were 
encouraged to provide ongoing feedback and thoughtful 
suggestions for ongoing course improvement. We took 
this feedback very seriously and it was critical to the 
development of CM As. 

Scale reliability 

Before examining survey results, it was important to 

TABLE 9. INTERNAL CONSISTENCY OF SURVEY 
INSTRUMENTS! 


Scale 

Time 

a 

STEBI-A (PSTE) 

Pre 

.93 


Post 

.85 

STEBI-A (STOE) 

Pre 

.79 


Post 

.75 

Laboratory Earth Orientation (LEO) 

Pre 

.85 


Post 

.90 

Beliefs About Science (BAS) 

Pre 

.62 


Post 

.68 


'as determined by Chronbach's Alpha 


TABLE 8. LABORATORY EARTH ORIENTATION (LEO) 
SAMPLE ITEMS _ 

1. I feel that I belong in this online course (Laboratory 
Earth) 

2.1 can talk/email to a course instructor or classmate if I 
have a problem with the course 

3. My classmates and instructors think carefully about 
what I have to say 

4. The course instructors and classmates respect my 
knowledge and experience 

5. The course instructors and my classmates care about 
my learning 

6.1 feel welcome in this course. 

'Participants used a five point Likert format (1 = low cohesion and 5 = 
high cohesion). See text for discussion. 

verify that these were psychometrically adequate scales. 
Since we were not able to conduct a factor analysis on 
scale data to establish that each was measuring a single 
construct (requiring at least 250 participants), we 
conducted a Cronbach's alpha analysis to evaluate the 
internal consistency of each scale. Cronbach's alpha is a 
coefficient (a number between 0 and 1) that rates the 
reliability of an instrument. Values above 0.70 indicate the 
scale is reliable. Pre- and post-course survey 
administrations provided two estimates of alpha for each 
scale. Results (Table 9) indicate the reliability estimates 
were adequate for the STOE and PSTE subscales of STEBI- 
A and the LEO. Reliability estimates were not adequate 
for the BAS. 

RESULTS 

Pre-post comparisons 

Scale totals were computed to be the mean item 
response across all items in each scale, before and after the 
Lab Earth 2 course. The significance level, alpha, (or the 
odds that the observed result is due to chance was), was 
set at a nominal level (.05) as per the conventional control 
for type-1 or false-positive errors. Given the standard 
deviations, pre- and post-scores were compared using a 
Wilcoxon t-test (a non-parametric test used when the 
population is not normally distributed) (see Table 10). 

Content knowledge assessment 

Participant teachers' content knowledge was 
measured by a series of 12 content questions. Teachers in 
Cohorts 1 and 2 (8-weeks) were encouraged to respond to 
the 12 content questions after each module (essentially 
they could take the test as many times as they liked). 
Teachers in Cohorts 3 and 4 (16-weeks) were directed to 
respond to the 12 content questions at the beginning and 
at the end of the course. In an attempt to understand 
possible differences in participants' overall content 
knowledge gain, we reviewed the highest content 
knowledge score of the 8-week participants (who were 
encouraged to take the test after each module) and the 
final content knowledge score for the 16-week 
participants. Before comparing the mean scores (with 24 
being the highest possible score), we ran an analysis of 
variance or ANOVA to ensure the differences between 
the groups (Cohort 1, 2, 3, and 4) and found there were no 
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TABLE 10. TOTAL COMPARISON OF PRE/POST-SURVEY MEASURES WITH WILCOXON T TEST 


Scale 

Pre 

Post 

Z 

Sig. 

Mean 

SD 

Mean 

SD 

STEBI-A (STOE) n = 51 

41.49 

5.251 

43.63 

5.067 

-2.583 

.010* 

STEBI-A (PSTE) n = 51 

48.18 

7.979 

50.96 

7.017 

-2.151 

.032* 

Laboratory Earth Orientation 

24.50 

2.915 

26.38 

3.009 

-3.551 

.000* 

Beliefs About Science 

17.72 

2.021 

18.42 

2.001 

-2.111 

.035* 


significant differences between the groups [F(3) = 2.146, p 
= .103, a = .05]. This implied that the scores were similar 
and supported our decision to combine the two 8-week 
cohort and the two 16-week cohort scores for an 
independent samples t-test to determine the impact of 
course duration. A Levene's Test showed that the 
homogeneity of variances assumption was met (F = .125, p 
-value = .725). Thus, we can assume that the data 
variances are equal (see Table 11). The t-test results 
suggest there was a significant difference (t = -2.439, p- 
value = .017, a = 0.05) between the 8- week course and the 
16-week course scores with the test scores on the 16-week 
course being significantly higher. 

To learn more about the 16-week participant content 
knowledge increase from pre- to post- test, we conducted 
a Wilcoxon-T-test. There was a significant difference [Z 
(22) = -4.014, p < .000, a = .05] from pre-test to post-test for 
the 16-week group. The post-test scores were significantly 
higher than the pre-test scores (see Table 12). The percent 
of increase for these teachers (when combined and 
calculated) was approximately 73%. 

Science teaching efficacy beliefs and enjoyment of 
science teaching 

On the Science Teaching Outcome Expectancy (STOE) 
subscale of the STEBI-A, there was a significant increase 
[Z(50) = -2.583, p = .010, a = .05] from pre-test to post-test. 
The effect size (Cohen, 1988) for this increase was (d 
= .419),), which is considered to be medium, and indicated 
that the STOE subscale had practical and statistical 
significance. One potential limitation to the use of STEB1- 
A for Lab Earth teachers is that it was designed for in- 
service teachers and a percentage (14%) of pre-service 
teacher participants may not have known how to respond 
to questions such as, "When teaching science, I generally 
welcome student questions." and "If parents comment 
that their child is showing more interest in science at 
school, it is probably due to the performance of the child's 
teacher." 

Even considering this limitation, these results imply 
that the Lab Earth 2 course had a significant impact on the 
teachers' belief that knowledge and performance in the 
classroom can positively impact their students' 
achievement. On the Personal Science Teaching Efficacy 
Beliefs (PSTE subscale) subscale of the STEBI-A, there was 
a significant increase [Z(50) = -2.151, p = .032, a = .05] 
from pre-test to post-test. These results imply that the Lab 


Earth 2 course had a significant impact on the teachers' 
personal science teaching efficacy. 

On the BAS scale, there was a significant difference [Z 
(50) = -2.111, p = .035, a = .05] between the pre-test and 
post-test scores. Although the statistical reliability of BAS 
is suspect, the result is consistent with formative feedback 
prompts that Lab Earth 2 had a favorable impact on the 
participant teachers' perspective on the extent to which 
they enjoy science teaching. 

Course cohesion 

On the LEO Scale there was a significant difference [Z 
(49) = -3.551, p < .000, a = .05] in pre-test and post-test 
scores. The effect size for this increase was considered to 
be medium (d = .634) and indicates that the LEO had 
practical as well as statistical significance. This implies 
that the Lab Earth 2 course had a significant impact on the 
participants' sense of belonging and community. 

DISCUSSION 

Participant teachers demonstrated significant 
increases on both subscales of the Science Teaching 
Efficacy Beliefs Instrument (STEBI-A) indicating overall 
increases in teachers' science teaching outcome expectancy 
(STOE) and personal science teaching efficacy (PSTE). 
Data also showed that teachers increased their sense of 
belonging to the Laboratory Earth course (LEO) and 
science teaching enjoyment (BAS). It seems that the Lab 
Earth 2 course successfully met our objectives. Further 
discussion of these results is presented according to our 
research questions. 

Content knowledge assessment 

These data presented support an affirmative response 
to our research question: "Does the Lab Earth approach 
achieve the cognitive goal of increasing teachers' 
knowledge and understanding of Earth science content?" 
These data indicate a significantly greater increase in 
content knowledge scores for Cohorts 3 and 4 than 
Cohorts 1 and 2. This increase in scores for the later 
cohorts might be due to a variety of factors including the 
use of the CMAs for assessment, length of the course, or 
participant demographics. 

An issue that the CMAs did resolve was our concern 
about motivational issues related to having teachers 
revisit the same questions repeatedly (at the end of each 
module) using content questions. Importantly, 


TABLE 11. HOMOGENEITY OF VARIANCES ON CONTENT KNOWLEDGE TESTS 


Scale 

8 Week Course 

16 Week Course 

T 

Sig. 

Mean 

SD 

Mean 

SD 

Content Knowledge Test 

12.00 

5.249 

15.25 

5.820 

-2.439 

.017* 
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TABLE 12. PRE- TO POST CONTENT KNOWLEDGE TEST COMPARISONS FOR 16-WEEK COURSE 


Scale 

Pre 

Post 

Z 

Sig. 

Mean 

SD 

Mean 

SD 

Content Knowledge Test 

10.23 

5.450 

17.68 

3.810 

-4.014 

.000* 


individualized responses to the teachers' CMAs material 
provided important opportunities for addressing 
misconceptions and encouraged accurate content 
knowledge - not just final grades or points. Table 13 
provides representative examples of our feedback. Given 
the possibility of 8 total points, teachers seemed satisfied 
with a score of 7 points and motivated to respond to 
feedback and re-do assignments until they achieved at 
least 7 points. CMA's also seemed to reduce teachers' 
grade anxiety by encouraging creative responses and 
direct learning applications (module by module). Some 
teachers took advantage of this assignment to create 
materials or invent processes they could also use in their 
own classrooms. 

What seemed most convincing, about the real benefit 
of the CMAs, was the teachers' expressed realization of 
their own increasing confidence and ability. As one 
teacher explained, "I feel myself becoming more confident 
in the content as I am completing this course. I'm learning 
more than just the basics and can help my students with 
the application of the science." 

Although CMAs are valued tools, an important task 
we are currently addressing is the refinement of the rubric 
to include more explicit references to Bloom's cognitive 
scale and connections to the learning objectives within 
each module. A review of exceptional products from the 
previous cohorts will be an important resource for adding 
more descriptive language to define these assignment 
expectations. It will be important to continue the Content 
Mastery Assignment (a constant measure) as a pre-to-post 
measure of improved content knowledge for the collective 
cohorts. 

In addition to overall content knowledge 
improvement, teachers expressed realization of the ways 
in which this course connected to their personal lives. 
One teacher realized, "I have taken a few online courses 
and all of them have been 'read this, write that and post a 
discussion.' With this class, we were connecting the 
content to our daily lives - how much water you use, what 
kind of soil is in your backyard." 

Teachers also noticed another connection - the ways in 
which the course activities connected to their classrooms. 


Teachers realized they were learning content that they 
could integrate directly into their own classrooms. One 
teacher indicated, "I'm teaching about geology at the 
moment and I am gathering all kinds of ideas as we 
progress." They viewed the course activities as a useful 
model and explained, "Everything is easy to take into the 
classroom." One participant summed-up her enthusiasm 
in all capital letters: "THE BEST THING IS THAT I AM 
GETTING SOME GREAT STUFF TO USE WITH MY 
OWN CLASSES." 

Although we implemented a grading system that 
focused on mastery of course material and worked to help 
teachers feel comfortable about exposing their areas of 
need by implementing an iterative grading system, a 
review of teachers' comments suggests they continued to 
worry about how they were being evaluated. Teachers 
worried about "getting the assignments done and keeping 
up on the discussion board" and "assignments [were] 
listed on the syllabus but not listed in the order of due 
dates." Some teachers wanted more "feedback on written 
assignments." While responses to teachers' management 
issues and concerns were clearly appreciated and helped 
to reduce anxiety, it was clear that teachers brought a 
grade expectation with them to this course. So, in fact, 
they were learning not to focus on their grades as they 
progressed through the course. 

Science teaching efficacy beliefs and enjoyment of 
science teaching 

These data support an affirmative response to our 
second research question, "Does the Lab Earth approach 
improve teacher's self efficacy and beliefs about their 
ability and confidence to teach science in the context of the 
Earth as a system?" Teacher participants showed 
significant overall increase from pre- to post-test on the 
STEBI-A sub-scales of Science Teaching Outcome 
Expectancy (STOE) and the Personal Science Teaching 
Efficacy Beliefs (PSTE). Although the BAS has some issues 
in the context of statistical validity, the results are 
consistent with STEBI results that teachers' enjoyment of 
teaching science improved as a result of their Lab Earth 
experience. 


TABLE 13. EXAMPLES OF FEEDBACK FROM THE INSTRUCTOR TO STUDENT ON CONTENT MASTERY 
ASSIGNMENTS _ 

"1 liked the format of your PowerPoint. You made good use of pictures. I would like you to re-visit and provide me 
with a few more details related to the factors that influence the availability of natural resources. I agree that supply and 
demand are critical, but both supply and demand have factors that influence them." 

"Your description of the movement of groundwater was concise and informative. I would have liked to have seen you 
provide more discussion of sustainability and the hydrologic equation." 

"Your approach is good, but it would be useful to re-visit the PowerPoints on plate tectonics and the rock cycle. Your 
definition of plate tectonics needs to be modified. The message I get from the slides is that pressure and temperature 
are the only link between the major rock groups. To link to plate tectonics, you could specify where the various rock 
types form." 


210 


Journal of Geoscience Education, v. 58, n. 4, September, 2010, p. 203-213 






Sense of Community 

The LEO data indicated that the participants felt that 
they were part of a community and that this sense of 
community was not necessarily a function of the time that 
the group was together in that there were no significant 
differences between the 8-week and 16-week cohorts. 
These results are consistent with feedback from the 
teachers about how the online environment was useful for 
interaction and exchanges. Teacher participants "enjoyed 
exchanging ideas with other class members" and the way 
in which the assignments and online discussions "really 
get a person thinking about our resources and how we 
take care of them." Some teachers had difficulty 
navigating the online discussion format, but in general 
teachers appreciated these discussions as a way to share 
ideas with other teachers. This certainly contributed to the 
creation of an overall sense of community. 

While teachers looked to the "creativity involved 
with the assignments and the encouragement to interact 
with each other and continue discussion about what we 
have worked on," they expected more discussion 
interactions with the course instructor. As one teacher 
explained, "In general I would love to hear from [the 
instructor] more. I realize he is busy . . . but more tidbits 
of knowledge on the discussion board would have been 
greatly appreciated." The teachers provided some 
additional ideas about how they might share more course- 
related resources via the discussion board. One teacher 
thought about exchanging "great examples of student 
work." Another teacher suggested, "Maybe [you could 
add] a special thread to the discussion board where we 
share lesson plan ideas, write-ups, useful websites - a 
thread for resources." So, in all, these teachers looked to 
the online environment as a way to connect with the 
course instructor and peer Lab Earth 2 teachers. 

Rapid response to formative evaluation 

We encouraged and responded to participant 
teachers' formative feedback throughout their course 
experiences. Participant teachers were ready with 
comments and suggestions about module activities and 
course design. There was a clear pattern of concern and 
confusion with the lesson modules from the onset of 
cohorts 1 and 2 that seemed to get smoothed out by the 
later cohorts. The instructor thoughtfully responded to the 
limitations and concerns raised by teachers - thus the 
participant teachers were actually helping to form the 
final course product cohort by cohort. 

Many teacher comments praised the course-provided 
PowerPoint presentations, websites, and videos. Some 
teachers raised concern about the difficulty of accessing 
needed information on the Internet. One teacher 
complained, "Internet searching is pretty difficult and 
time consuming sometimes." Another remarked, "I'm not 
very good with fishing the Internet." Teachers seemed to 
have the most trouble with the "USGS site for 
hydrographs." 

An important piece of feedback that we are 
considering as we move forward with the Lab Earth 
approach is the pacing of an online course for teachers. 
Teachers identified their frustrations with managing 


assignment deadlines that fell during busy school times 
(such as parent teacher conferences and school holidays). 
One teacher suggested that the instructor "give more time 
between due dates." Another teacher prompted, "Decide 
what's really important for us to do as busy teachers." 
There seemed to be a particular problem with the course 
assignments in the final module. Teachers explained, 
"Some of the directions were a bit vague and hard to 
follow. Others complained, "The last two modules have 
involved a lot more time." and "This is a terrible time of 
year to increase the workload." While teachers conferred 
harsh judgment on the pace and expectations of the 
course, calendar demands (i.e. Parent Teacher 
Conferences) are important realities in a teachers' world 
that need to be considered during the design of on-line 
courses offered during the academic year. 

IMPLICATIONS FOR TEACHING 

Participating teachers demonstrated significant 
increases on both the Science Teaching Efficacy Beliefs 
Instrument (STEB1-A) and personal science teaching 
efficacy (PSTE) meaning they feel more comfortable in 
their individual ability to teach science. As stated above, 
one teacher explained, "I feel myself becoming more 
confident in the content as I am completing this course. 
I'm learning more than just the basics and can help my 
students with the application of the science." 

With each of the modules, teachers created projects 
and completed activities that they could alter to fit their 
own curriculum immediately. Participants also had the 
valuable experience of being able to share ideas with other 
teacher participants and solve problems surrounding the 
implementation and execution of the new ideas. 

Perhaps most importantly, our results imply that the 
Lab Earth 2 course had a significant impact on the teachers' 
belief that their own knowledge and performance in the 
classroom can positively impact their students' 
achievement. 

CONCLUSIONS 

The Laboratory Earth series is an example of on-line 
courses that use contemporary pedagogy and technology 
to improve content-knowledge and attitudes of teachers 
toward science. This type of effort is important in that 
educators, especially those who teach K-8 students, play 
an important role in providing students with the 
foundation for their success in scientific disciplines. 
Although the data represent a relatively small sample of 
teachers, these data gathered over a long duration (four 
iterations of the same course as guided by systematic data 
collection, reflection, and analysis) indicate that the 
approach used to design, implement, and deliver 
Laboratory Earth resulted in overall improvement of 
teachers' content knowledge and attitude towards science. 
The results support the conclusion that appropriately 
designed on-line course work can be an important 
component of a long-term effective professional 
development portfolio that deepens teachers' knowledge, 
sharpens teachers' skills, and maintains teachers' 
knowledge of science developments (Harwell, 2003, 
CORD, 2001, and NCMST, 2000). 
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Additional measures of improved teachers' ability 
might include student achievement data such as student 
test scores, teacher-created student evaluations, 
observation instruments, student work samples, or action 
research plans. One pre/post-test possibility would be to 
have teachers review/evaluate a lesson plan or a student 
work sample at the onset and completion of the course. 
This might provide a measure of improved teaching 
ability, resourcefulness, and content connections. 
Teachers might be asked to complete a grading rubric on 
one student sample and make recommendations for re¬ 
teaching or improving the lesson plan. This new approach 
would help to direct participant teachers to apply their 
new Earth system knowledge in the design of student 
learning experiences. 

Continuous feedback from participants and reflection 
on and modification of our educational approach resulted 
in an iterative grading process that uses content mastery 
assignments as its foundation. CMAs provide an 
assessment of teachers' content knowledge and ability to 
make connections at higher cognitive levels. CMAs also 
provided an important opportunity for teachers to 
develop their metacognitive skills to plan their approach 
to completing the task, monitor their learning in the 
context of their plan as the module progresses, and 
evaluate their level of understanding after the completion 
of the task based on instructor feedback. 

Laboratory Earth provides another example of how on 
-line programs, if designed appropriately, are a win-win 
situation for institutions of higher education and the 
teachers they serve. For the teachers, online professional 
development provides convenience that traditional face-to 
-face professional development may not allow. For 
institutions of higher education, online courses can serve a 
larger market of professionals who want and need access 
to the academic resources. 
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